OpenAI has been surprising the world with its achievements lately. With ChatGPT, an AI-powered chatbot, the company has made it possible to create interaction-style conversations. One thing about ChatGPT that has everyone on the edge of their seats is its human-like responses. Another fact fueling the excitement about OpenAI’s latest venture is that it is available for free. While everyone can access ChatGPT for free, OpenAI is bearing a hefty weight in order to keep ChatGPT up and running.
Netizens were quick to find clues regarding the expenses that OpenAI has to bear to keep ChatGPT running the way it already does. Some hints about the development and running of such an AI service are shocking, to say the least. For instance, the cost of running ChatGPT is $100,000 per day. According to multiple sources, Microsoft’s Azure cloud is hosting ChatGPT so that OpenAI does not have to invest in a physical server room. Considering Microsoft’s current rates, it is $3 an hour for a single A100 GPU, and each word generated on ChatGPT costs $0.0003. At least eight GPUs are in use to operate on a single ChatGPT. So, when ChatGPT generates an average response of 30 words, it will cost nearly 1 cent for the company. Through such an estimation, OpenAI could be spending at least $100K per day or $3 million monthly on running costs.
Another source claims that a single NVIDIA A100 GPU can run a 3-billion parameter model in about 6ms. Considering this speed as the reference point, a single NVIDIA A100 GPU could take 350ms seconds to print out just a single word on ChatGPT. Since the latest version, ChatGPT-3.5, has over 175 billion parameters, it will need around five A100 GPUs to perform the necessary action to get an output for a single query. The average number of at least 8 A100 GPUs is derived after taking ChatGPT’s capability of outputting around 15-20 words per second.
ChatGPT can do everything from writing complex Python code to crafting college-level essays on demand. All it needs is prompt. Owing to this, concerns about technology taking over jobs such as programmers and writers have been raised. As earth-shattering as this AI chatbot seems, it is evident that ChatGPT does not hold all the answers. Admittedly, the world’s most advanced AI chatbot is trained on models created in 2021 or before that. Thus, there is something noticeably lacking in the responses it creates for each query.
Despite this minor setback, ChatGPT has garnered over a million users within a few days of the official launch. Even though most of this large number of users are not active users, it is undeniable that OpenAI’s ChatGPT has stirred enough interest in people in a very short time. Considering the fact that the masses are intrigued by OpenAI’s latest offering, the company has a long way to go if it needs to turn ChatGPT into an AI tool that is thriving.
